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' REMARKS 

The following references were used alone or together to 
reject claims 1-14, 16, 17, 19-28, and 31-40: 

Sugiyama et al. (U. S . 6, 002, 403, hereinafter Sugiyama); 
Marrin et al. (U.S. 5,808,613, hereinafter Marrin); 
Matsuda (U.S. 6,346,956, hereinafter Matsuda) ; 
Miller (U.S. 6,229,542, hereinafter Miller) ; and 
Horvitz et al. (U.S. 5,880,733). 

None of .these references shows or suggests the 
inventions of independent claims 1, 20, 31 or 37. As such, all 
of the pending claims are patentable over these references taken 
alone or in any combination. 

Independent claim 1 provides a method of generating a 
display. The method includes displaying a three-dimensional 
environment and at least two tasks in the environment. Each task 
is capable of including an image of at least two windows. 
Movement of one of the tasks is displayed in response to input 
from the user. 

Independent claim 20 provides a computer-readable 
medium with components that are able to display a three- 
dimensional environment and at least two tasks, each containing 
images of at least two windows. A move task component moves one 
of the task images in response to input from a user. 

Independent claim 31 provides a method of generating a 
display that includes displaying a non-focus task capable of 
including an image of at least two windows and displaying a stage 
area in a three-dimensional environment. The non-focus task is 
moved to the stage area based on input from the user and becomes 
a focus task such that the user can manipulate at least a portion 
of one window. 

Independent claim 37 provides a computer-readable 
medium with components that display a three-dimensional 
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environment having a stage and at least one task containing 
images of at least two windows. A movement component moves the 
task toward the stage and a conversion component converts the 
task into a focus task when the task reaches the stage. 

None of these claims is shown or suggested in the cited 
art because none of the cited art shows a task that has an image 
of at least two windows and that is shown as moving in the three- 
dimensional environment. In particular, Sugiyama does not show a 
task that includes these limitations. 

In the Office Action, it was asserted that Sugiyama 
showed a task containing an image of at least two windows in 
FIGS. 5 and 7. In particular, A/V wall 112 and application wall 
113 were asserted to be tasks. 

However, walls 112 and 113 do not include images of at 
least two windows. Instead, each wall consists of displayed 
buttons used to activate applications. Since these walls do not 
show at least two windows, Sugiyama does not show or suggest the 
invention of claims 1, 20, 31, or 37. 

By providing images of at least two windows in a task, 
the present invention allows users to recognize a task based on 
the windows displayed in the task. For instance, a task that 
involves writing a letter regarding a bill could include a window 
for a word-processing application and a window for a billing 
application. While looking at the three-dimensional environment, 
the user is able to find this task based on the fact that the 
task includes an image of both the word-processing window and the 
billing window. Further, the contents of the windows can help 
the user to select between two tasks that may use the same 
application. Thus, by looking at the content of the windows, the 
user is able to discern which task contains the data they want. 

The invention of claims 1, 20, 31, and 37 also allow a 
user to move two windows with a single command. Thus, by issuing 
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a command to move a task, users are able to move all of the 
window images associated with that task. None of the cited art 
shows or suggests the ability to move more than one window by 
moving a task. 

Since none of the cited references show or suggest the 
invention of claims 1, 20, 31 or 37, claims 1-41 are patentable 
over the cited art. 

CLAIM 11 

Claim 11 was rejected under 35 U.S.C. §103 (a) as being 
obvious from Sugiyama in view of Marrin and Horvitz. Claim 11 
depends from claim 1 and includes a limitation to removing a 
movement control based on a signal from a touch-sensitive input- 
device that indicates that the user is not touching the input 
device . 

None of these references shows or suggest removing a 
movement control based on input from a touch-sensitive device 
indicating that a user is not touching the input device. 
Although Horvitz does mention a touch sensitive pad, he does not 
indicate that a movement control should be removed from the 
display when the user is not touching the pad. Marrin and 
Sugiyama also fail to show or suggest removing a control when a 
user is not touching an input device. 

Note that the invention of claim 11 allows users to 
quickly dismiss the virtual camera control simply by releasing an 
input device. This makes it easier for the user to reduce the 
clutter on the display and is especially helpful when the user is 
not navigating the camera through the environment. 

Since none of the cited references show or suggest 
removing a control when a user is not touching an input device, 
their combination does not show or suggest the invention of claim 
11. 
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Claim 16 

Claim 16 was rejected under 35 U.S.C. §102 (e) as being 
anticipated by Sugiyama. Claim 16 depends from claim 1 and 
includes a further limitation wherein displaying movement of a 
task includes displaying movement of the task along a surface. 

Sugiyama does not show or suggest such movement. In 
the Office Action it was asserted that FIGS. 5 and 7 showed such 
movement. However, these figures simply show two different 
display modes. In the mode of FIG. 5 the A/V wall is shown in 
front of the user. In the mode of FIG. 7, the application wall 
is shown in front of the user. These figures do not show 
movement along a surface. They simply indicate that the images 
on the various walls are switched when the user switches modes. 

In claim 16, the task moves along a surface. Since 
such movement is not shown or suggested by Sugiyama, claim 16 is 
patentable over Sugiyama. 

Claim 19 

Claim 19 was rejected under 35 U.S.C. §102 (e) as being 
anticipated by Sugiyama. Claim 19 depends from claim 1 and 
includes a limitation to displaying a menu with a task movement 
selection and displaying movement of a task based on the user 
selecting the task from the menu. 

Sugiyama does not show or suggest displaying a menu 
with a task movement selection. In the Office Action it was 
asserted that col. 9 lines 8-11 of Sugiyama showed such a menu. 
However, this section merely states that when the user clicks on 
an empty space above the center window, the view will change to 
show a Net Space 114 located above the center window. This 
section does not show or suggest a menu with a task movement 
selection. 

Since Sugiyama does not show or suggest a menu with a 
task movement selection, it does not anticipate claim 19. 
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Claim 33 

Claim 33 was rejected under 35 U.S.C. §102(e) as being 
anticipated by Sugiyama. Claim 33 depends from claim 31 and 
includes a further limitation wherein a focus task is converted 
into a non-focus task by capturing an image of the focus task and 
replacing the focus task with the image. 

Sugiyama does not show or suggest a step of capturing 
an image of a focus task or replacing a focus task with an image 
to form a non-focus task. In the Office Action, it was asserted 
that FIGS. 5 and 7 showed these steps. However, Sugiyama never 
discusses capturing an image of any of the walls shown in FIGS. 5 
or 7 or of replacing the contents of one of the walls with an 
image. In fact, in Sugiyama, each wall contains active objects, 
not simply an image. This is evident from the fact that it is 
possible to double-click on a button on the left or right wall in 
Sugiyama. Because of this, it is clear that Sugiyama is not 
replacing the focus task with an image of the focus task to form 
a non-focus task. Instead, Sugiyama only suggests moving the 
contents of one wall to another wall. 

Since Sugiyama does not show or suggest capturing an 
image of a focus task or replacing a focus task with its image, 
it does not show or suggest the invention of claim 33. 

Claim 34 

Claim 34 was rejected under 35 U.S.C. §103 (a) as being 
obvious from Sugiyama in view of Marrin. Claim 34 depends from 
claim 33 and includes a limitation wherein capturing an image 
includes moving a camera from a current position to a preferred 
position, rendering the image of the focus task from the 
preferred position, storing the image, and returning the camera 
to the current position. 

Neither Sugiyama nor Marrin suggest these steps for 
capturing an image of a focus task. In the Office Action, it was 
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asserted that Marrin showed these steps at col. 3, lines 60-67, 
and col. 5, lines 34-38. However, the cited sections simply 
discuss moving through a three-dimensional environment based on 
input from a user. They do not discuss capturing an image of a 
focus task by storing a rendering of the focus task taken from a 
preferred position in the three-dimensional environment. 

Note that the process of claim 34 ensures that a 
consistent view of the focus task will be used when capturing an 
image of the focus task before replacing the focus task with the 
image. This means that regardless of the virtual camera's 
position in the environment when the focus of the tasks changes, 
the captured image of the focus task will be taken from the 
preferred position. Neither Sugiyama nor Marrin show or suggest 
the desirability of this behavior or a mechanism to perform these 
steps . 

As such, claim 34 is additionally patentable over 
Sugiyama and Marrin. 

Claim 35 

Claim 35 was rejected under 35 U.S.C. §103 (a) as being 
obvious from Sugiyama in view of Marrin. Claim 35 depends from 
claim 33 and includes a limitation wherein capturing an image of 
the focus task involves displaying an image of the environment 
from a current position while a virtual camera is moved to a 
preferred position, an image of a focus task is rendered and 
stored, and the virtual camera is returned to the current 
position. 

Neither Sugiyama nor Marrin show or suggest maintaining 
an image of an environment while a virtual camera is moved to a 
preferred location to render and store an image of a focus task. 

As such, claim 35 is additionally patentable over Sugiyama and 
Marrin. 

Claim 36 
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Claim 36 was rejected under 35 U.S.C. §103 (a) as being 
obvious from Sugiyama in view of Marrin. Claim 36 depends from 
claim 31 and includes a limitation to displaying a menu that 
includes a task selection for the non-focus task, selecting the 
task selection based on input from a user, and moving a virtual 
camera to bring the non-focus task into view based on the 
selection . 

Neither Sugiyama nor Marrin show or suggest selecting 
an entry in a menu to bring a non-focus task into view. In the 
Office action, it was asserted that col. 9, lines 8-11 showed a 
step of displaying a menu, and that FIG. 10 showed a step of 
brining a non-focus task into view based on a selection made in a 
menu. However, the cited section of col. 9 does not show or 
suggest the display of a menu or the selection of an entry in a 
menu. Instead, it merely states that a Net Space 114 can be 
brought into view by clicking on the space above the walls shown 
in FIGS . 4A, 5, and 6A. Furthermore, FIG. 10 does not show a 
menu or a selection of a menu entry. 

Since neither Sugiyama nor Marrin show or suggest 
displaying a non-focus task in response to a user selecting an 
entry in a menu, claim 36 is patentable over this combination of 
references . 

Claim 40 

Claim 40 was rejected under 35 U.S.C. §103 (a) as being 
obvious from Sugiyama in view of Miller and Marrin. Claim 40 
depends from claim 37 and includes a limitation to generating an 
image of a focus task by moving a virtual camera to a preset 
location and rendering the appearance of the three-dimensional 
environment from that point of view. The image of the focus task 
then replaces the focus task. 

None of the cited references show or suggest a step of 
moving a virtual camera to a present location to capture an image 
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of three-dimensional environment to generate an image of a focus 
task that will used in place of the focus task. In the Office 
Action, it was asserted that Sugiyama taught a step of generating 
such an image of a focus task at col. 6, lines 11-13. However, 
this section does not show or suggest capturing an image of a 
focus task so that the image can be used in place of the focus 
task. 

As such, claim 40 is patentable over Sugiyama, Marrin 
and Miller. 

Conclusion 

In light of the above remarks, claims 1-41 are 
patentable over the cited art. Reconsideration and allowance of 
the claims is respectfully requested. 

The Director is authorized to charge any fee deficiency 
required by this paper or credit any overpayment to Deposit 
Account No. 23-1123. 

Respectfully submitted, 
WESTMAN, CHAMPLIN & KELLY, P. A. 

By: 

Theodore M. Magee, AKeg. No. 39,758 

Suite 1600 - International Centre 

900 Second Avenue South 

Minneapolis, Minnesota 55402-3319 

Phone: (612) 334-3222 Fax: (612) 334-3312 
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MARKED -UP VERSION OF REPLACEMENT CLAIMS 

37 . (Amended) A computer-readable medium having computer- 
executable components comprising: 

a environment display component capable of displaying a 
three-dimensional environment on a computer 
screen, the three-dimensional environment 
comprising at least one stage and at least one 
non-focus task comprising images of at least two 
* windows ; 

a movement component capable of displaying animated 

movement of a non-focus task toward a stage; and 
a conversion component capable of converting the non- 
focus task into a focus task when the non-focus 
task reaches the stage. 



- Summary of Video Entitled CHIMP System 

by. Mark Mine of the University of North Carolina, 1.996 



Applicants have recently been made aware of a video 
produced by Mark Mine in 1996. This video shows a split 
screen with one portion of the screen containing a depiction 
of a user wearing a head mounted display and manipulating 
two input devices, one in each hand. In the other portion of 
the display, a three-dimensional environment which is 
presumably being displayed to the user through the head- 
mounted display is shown. The video depicts three- 
dimensional objects located in the three-dimensional 
environment and the user manipulating, the position and 
orientation of those objects. Using a sweeping gesture with 
the left hand moving to the side of the user" and then 
forward, a three-dimensional tool space is brought into 
view. Initially the tool space contains a flat panel with 
control buttons. The user then makes a flipping motion with 
their left hand and flips the tool space to expose the 
underside of the .panel. The underside of the panel is 
populated with three-dimensional objects that are positioned 
within squares on the tool space plane. The user positions a 
blue selection dot on one of the three-dimensional objects, 
selects the object and drags the object into the three- 
dimensional space'. The user then positions the three- 
dimensional object within the three-dimensional environment. 
The user "' then makes a motion to their left-hand side with 
their left hand. This results in an animation in which the 
tool space, rotates out of view. 



